Mining Cross-Person Cues for Body-Part Interactiveness Learning in HOI Detection

Human-Object Interaction (HOI) detection plays a crucial role in activity understanding. Though significant progress has been made, interactiveness learning remains challenging problem HOI detection: existing methods usually generate redundant negative H-O pair proposals and fail to effectively extract interactive pairs. studied both whole body- part- level facilitates the pairing, previous works only focus on target person once (i.e., local perspective) overlook information of other persons. In this paper, we argue that comparing body-parts multi-person simultaneously can afford us more useful supplementary cues. That said, learn body-part from global perspective: when classifying person’s interactiveness, visual cues are explored not herself/himself but also persons image. We construct saliency maps based self-attention mine cross-person informative holistic relationships between all body-parts. evaluate proposed method widely-used benchmarks HICO-DET V-COCO. With our new perspective, global-local achieves improvements over state-of-the-art. Our code is available at https://github.com/enlighten0707/Body-Part-Map-for-Interactiveness .